Repository navigation
fix(resharding): use replica_session role and disable FK validation - #1540
Merged
Merged
Conversation
meskill
force-pushed
the
meskill-2026-09-14-test-resharding---yry
branch
from
September 14, 2026 12:09
7808268 to
245b83e
Compare
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
meskill
force-pushed
the
meskill-2026-09-14-test-resharding---yry
branch
from
September 14, 2026 15:13
245b83e to
3a03717
Compare
meskill
force-pushed
the
meskill-2026-09-14-test-resharding---yry
branch
from
September 14, 2026 15:43
3a03717 to
d84260b
Compare
meskill
changed the title
meskill
force-pushed
the
meskill-2026-09-14-test-resharding---yry
branch
from
September 14, 2026 22:43
c7722dc to
a09fed1
Compare
meskill
force-pushed
the
meskill-2026-09-14-test-resharding---yry
branch
from
September 15, 2026 12:14
0b305d6 to
1910ee7
Compare
Contributor
Author
|
Added check for destination pg versions to avoid possible issues with FK NOT VALID for partitioned tables since it's only supported from pg18. See https://www.postgresql.org/docs/17/sql-altertable.html#SQL-ALTERTABLE-DESC-ADD-TABLE-CONSTRAINT and https://www.postgresql.org/docs/18/release-18.html#RELEASE-18-CONSTRAINTS |
pull Bot
pushed a commit
to TheTechOddBug/pgdog
that referenced
this pull request
Sep 29, 2026
Add a new PostDataValidation phase for schema_sync that allows to run the validation after copying the data. Previously in pgdogdev#1540 the FK constraints where added with `NOT VALID` option marking the flag `convalidated = false` for the related table, which disables the check on creation for existing data and maybe disables some optimization that could be done by db for this table. New step runs `VALIDATE CONSTRAINT` on this FK to make sure the `convalidated` is set to true on table and the db actually will start to think the values are validated. Like this since it should not block the table from actual writes comparing with creating the FK without options, so we can run this in parallel to our replication process. The check is manual for now and later PRs will use them to run during validation. For the remainder of the flow: - we copy tables with COPY command - we create FK constraints with `NOT VALID` since the copied data could be not in sync because different tables were copied at different snapshot time - we replicate changes - eventually, data "should" become synchronized and valid - the `PostDataValidation` could be run after the replication or after some time during the replication to see if there any errors.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Handle the FK constraints during resharding process:
When we copy tables in parallel we make the snapshots of tables at the specific moment of time and because of this we can have tables on destination with different WALs (that's why we save lsn for tables after copy) and that's could make data inconsistent for example for FK and other cross-table constraints.
The error could happen in both cases: when we do schema sync for the constraints (they may fail on apply), or during the replication (that may fail because part of the transaction could be already in some table snapshot, but not another table snapshot).
To resolve this:
SET session_replication_role=replicaon destination in order to disable rules and triggers during the update. That requires additional specific GRANT or superuser, so an error handling for this with a little explanation was added.NOT VALIDfor FK during replication, so the FK won't be triggered when added (the role doesn't disable it automatically).This basically disables this validations to the sake of assumption that we have consistent data on source and we do the proper copy that maintains the consistency so during the replication we'll eventually reach the consistent state.